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PROFIT  MAXIMIZATION  MODELS  FOR 


EXPONENTIAL  DECAY  PROCESSES 
by 

S.P.  Sethi,  G.L.  Thompson  and  V.  Udavabhanu 

ABSTRACT 

''''''^■A  number  of  real  world  processes  can  be  modelled  as  exponential  decay 
processes.  Examples  are:  machine  replacement,  oil  well  extraction,  advertising 
goodwill,  repair  and  cleaning  activities,  etc..  In  this  paper  we  analyze  a 
series  of  discounted  or  undiscounted,  deterministic  or  stochastic  exponential 
decay  models.  We  characterize  finite  and  infinite  horizon  optimal  solutions 
for  each  model.  We  show  that  the  solution  can  be  characterized  for  the  oil 
drillers  problem  in  the  following  way:  Once  a  well  of  sufficient  capacity  is 
drilled,  oil  is  pumped  from  it  until  the  oil  remaining  decreases  to  a  fixed 
cut-off  level;  then  the  well  is  abandoned,  and  a  new  well  is  drilled.  The 
resulting  process  when  repeated  over  time  appears  to  be  the  same  as  an  oil 
source  which  produces  oil  revenue  continuously  at  the  fixed  cut-off  level.  In 
other  words,  the  excess  revenue  received  from  an  oil  well  when  its  capacity 
is  greater  than  the  cutoff  level  is  just  sufficient  to  pay  for  drilling  costs 
for  a  new  well._ 
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1.  Introduction 


Several  real-world  processes  are  subject  to  the  phenomenon  of  exponential 
decay.  Here  we  analyze  various  optimization  models  involving  such  processes. 

Consider,  as  a  first  example,  a  machine  replacement  problem.  The  perform¬ 
ance  level  of  the  machine  -  which  may  be  measured  in  terms  of  the  output  quantity 
and  the  production  rate  -  can  be  assumed  to  decrease  exponentially  with  time. 

The  machine  earns  revenue  during  its  lifetime,  and  then  must  be  replaced  at 
a  certain  cost.  The  question,  therefore,  is  to  decide  on  the  timing  of  trachine 
replacement,  and  the  capacity  (in  terms  of  initial  performance  level)  of  the 
new  machine  [ 2 ,  6 ,  7 ] . 

Another  example  concerns  the  determination  of  the  optimal  drilling  times 
and  capacities  of  oil  wells  [7],  The  rate  of  extraction  of  oil  from  the  well 
may  be  assumed  to  be  directly  proportional  to  the  quantity  of  oil  remaining  in 
the  well,  which,  in  turn,  implies  that  the  process  is  subject  to  exponential 
decay.  Profits  are  generated  during  the  course  of  oil  extraction;  at  some 
point,  the  well  is  abandoned,  and  a  new  one  drilled.  The  same  model  is  repre¬ 
sentative  of  mining  operations  in  general. 

Memory  is  yet  another  process  subject  to  exponential  decay;  consequently, 
our  models  may  be  used  to  formulate  an  advertising  policy  problem  [4,  5] .  Media 
commercials  are  aimed  at  generating  goodwill  in  the  minds  of  consumers,  leading 
to  enhanced  sales.  The  loss  of  goodwill,  as  a  consequence  of  the  decline  in 
memory,  has  to  be  countered  by  exhibiting  further  commercials;  the  timing  and 
quantity  of  commercials  become  the  decision  variables. 

The  periodic  repainting  of  inns  [3],  and  the  cleaning  of  swimming  pools, 
which  provide  recreational  benefits  to  society  at  large,  are  further  examples  of 
situations  which  fit  into  the  general  framework  of  this  paper. 
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Section  2  lays  down  the  preliminary  groundwork  for  the  models;  Sections 
3,  4,  and  5  analyze  the  deterministic  discounted,  deterministic  undiscounted, 
and  stochastic  models  respectively. 

2.  The  Oil  Driller's  Problem 

We  now  present  several  models  involving  exponential  decay,  framed  in 
terms  of  the  oil  driller's  problem.  We  assume  that  at  any  instant,  there  is 
at  most  one  functioning  oil  well.  The  rate  of  oil  extraction  is  given  to  be 
a  constant  D  times  the  quantity  of  oil  C(t)  remaining  in  the  well  at  time  t. 
The  oil  is  sold  at  a  unit  profit  of  P.  As  the  oil  reserves  get  depleted,  the 
rate  of  extraction  eventually  decreases  to  uneconomic  levels,  making  it  worth¬ 
while  to  abandon  the  well  and  drill  a  new  one  at  a  cost  of  f(v),  where  v 
is  the  capacity  of  the  well.  The  function  f(v)  is  assumed  to  be  continuous 
and  twice  differentiable.  We  may  further  suppose,  without  loss  of  realism, 
that  v  *  0,  f(0)  *  0,  and  f'(v)  >  0. 

The  rate  of  oil  extraction,  then,  is  represented  by 

C(t)  -  -DC(t).  (1) 

Using  the  boundary  condition 

C(0)  -  v,  (2) 

the  solution  to  (1)  may  be  obtained  as 

C(t)  -  ve~Dt,  (3) 

where  time  is  measured  from  the  instant  at  which  the  well  is  drilled.  Drilling 
is  assumed  to  be  instantaneous. 
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To  complete  the  model,  we  specify  the  objective  function: 


Maximize  J  ■  I 

Vei  1 


i+1 


/  PDvie~D(t“ti)dt  -  f(vt) 


«-  Ci 


(4) 


The  drilling  times  t^,  and  the  corresponding  well  capacities  v^»v2>‘ 

constitute  the  decision  variables. 


3.  Deterministic  Discounted  Models 

In  this  section,  we  shall  analyze  a  deterministic  infinite  horizon  model, 
with  a  continuous  discount  rate  r  >  0. 

It  is  clear  by  symmetry  that  the  interval  T  between  drillings  will  re¬ 
main  constant;  so  will  the  capacity  v  of  the  oil  well;  see  Figure  1.  Note  its 


Figure  1.  Oil  well  capacity  over  time. 

similarity  to  the  simple  EOQ  lot  size  model.  If  J  represents  the  infinite 
horizon  profit  function,  we  may  write 

T 

J  -  /  PDve'Dte_rtdt  -  f(v)  +  Je' 

0 


(5) 
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which  may  be  more  conveniently  expressed  as 


J 


/i  -rT. 

(1-e  ) 


PPg-e-(I>fr)T) 

(D+r) 


f(v) 

v 


(6) 


The  objective  is  to  maximize  J  with  respect  to  v  and  T.  Further  analysis 
depends  on  the  form  of  the  cost  function;  we  consider  two  cases. 


Case  1:  f"  (v)  ^  0. 


Here  f(v)  is  concave  in  v,  so  that  ~~~  Is  nonincreasing  with  respect 
to  v.  A  cursory  examination  of  (6)  makes  it  patent  that  J  is  maximized  by 
letting  v  become  as  large  as  possible.  Hence  we  need  an  upper  bound  on  the 
drilling  capacity;  we  assume  that  v  s  V  <  °°. 

For  the  reason  just  mentioned,  the  optimal  drilling  capacity  must  be  at 
the  upper  bound,  provided  that  it  yields  a  positive  profit.  Thus  we  may  write 

f  V  if  PDV(l-e~{I>+r)T*)  >  (D+r)  f  (V) 


* 

V  = 


(7) 


0  otherwise 


The  optimal  drilling  interval  is  obtained  by  differentiating  J  with 
respect  to  T  and  equating  it  to  zero;  hence  T*  is  determined  from  the 
relation 


which  is  valid  under  the  hypothesis  that  v  ■  V.  Any  solution  to  (8)  yielding 

T*  s  0  implies  that  v  *  0;  further,  T*  ■  *  only  if  PDV  *  (D+r)f(V),  in 

* 

which  case,  we  must  have  v  *  0,  from  (7). 


r 
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Case  2:  f"(v)  >  0. 

Here  f(v)  is  strictly  convex  in  v,  so  that  is  an  increasing 

function  of  v.  This  property  of  decreasing  returns  to  scale  eliminates  the 

£ 

need  for  an  upper  bound  on  v  so  that  v  will  always  be  finite. 

Equating  to  zero  the  derivatives  of  J  with  respect  to  v  and  T,  the 

•k 

optimal  values  v  and  T*  are  obtained  as  the  simultaneous  solutions  to  the  equations 


k 

f  (v  ) 


PD(1  -  e-^1*) 
(IH-r) 


(9) 


and 


rT*  , 
e  -  1 

(IH-r)T*  . 
e  -  i 


r 

(IH-r) 


i  .  . 

A  *  * 
v  f'(v  ) 


(10) 


We  now  show  that  it  is  always  optimal  to  drill  a  positive  quantity. 

rfr 

Proposition  1:  v  >0. 

Proof :  Using  (9),  the  profit  function  (6)  may  be  evaluated  at  the  optimum  as 


* 

J 


1 


1-e 


-rT* 


v  f ’ (v*) 


-  f(v*) 


]• 


(11) 


Now  strict  convexity  of  f(v)  implies  that  f’(v)  >  ;  further,  since 

f'(v)  >  0  by  assumption,  it  may  be  deduced  from  (9)  that  T*  >  G.  Consequently, 
(11)  implies  that  J  >0,  from  which  we  may  infer  that  v  >0. 

Q.E.D. 


Equations  (9)  and  (10)  may  be  employed  to  further  simplify  (11) ,  yielding 

(12) 


*  -DT* 
PDv  e 


I 

J 
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It  is  easy  to  infer  from  (12)  that  J  <  »  and  T*  <  ”.  Thus  v  ,  T*,  and  J 
are  strictly  positive  and  finite-valued.  In  Figure  2  we  have  sketched  the 
optimal  solution.  Equation  (12)  affords  an  interesting  interpretation. 


Figure  2.  Optimal  oil  extraction  rate  over  time. 


Notice  that 


,  *  -DT*  -rt 

/  PDv  e  e  dt 

0 


*  -DT* 
PDv  e 


Thus  J  is  the  infinite  horizon  discounted  profit  which  would  be  obtained  by 

*  -DT* 

selling  oil  at  the  constant  rate  of  Dv  e  ,  with  no  drilling  costs;  further, 

this  constant  rate  is  precisely  the  rate  of  oil  extraction  just  at  the  instant 

* 

before  the  well  is  abandoned  in  favor  of  a  new  one.  In  Figure  2,  J  is  the 
discounted  value  of  the  shaded  area.  Therefore  the  discounted  value  of  the  cross 
hatched  area  equals  the  discounted  value  of  the  drilling  costs. 


Example : 


Suppose  D  ■  1,  r  ■  0.1,  P  =  100 
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Case  1:  f(v)  =  50v,  V  =  40. 

Equation  (8)  reduces  to 


l.le 


-T* 


— 1. IT* 
e 


0.045, 


yielding  T*  =  1.759, 

* 

so  that  v  =  40  from  (7). 

Case  2:  f(v)  =  50v^. 

Equation  (10)  reduces  to 


0. IT*  1. IT* 
22e  -  e 

which  yields  T*  *  1.236. 
Equation  (9)  reduces  to 


21, 


* 


v 


yielding 


=  0.909(1  -  e'1*11*), 
* 

v  »  0.676. 


4.  Deterministic  Undiscounted  Models 


4.1  Finite  Horizon  Models: 

Let  T  be  the  horizon  length,  and  n  the  number  of  drillings;  then  the 
objective  is  to 


Maximize  J 


v'i 


n 

Z 

i=l 


'i+1 


PDv.  e 
x 


-D(t-ti) 


dt 


-  f(vt) 


(13) 


where  t^  *  0  and 


n+1 


Equation  (13)  may  be  rewritten  as 

■D(ti+rti). 


n 

Z 

i-1 


Pv1(l  -  e 


f(vt) 


(14) 


We  need  to  analyze  two  cases,  as  before. 
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Case  1;  f"(v)  sO. 

As  in  Section  3,  the  concavity  of  f(v)  implies,  from  (14),  that  J 

n 

is  maximized  by  letting  become  infinitely  large;  so  we  assume  that 

v.  sV  <  »  . 

First  suppose  that  only  a  single  well  may  be  drilled  in  the  interval  [0,T]. 
For  the  afore-mentioned  reason,  the  optimal  drilling  capacity  must  be  at  the 
upper  bound,  provided  that  it  yields  a  positive  profit.  Thus 

fv  if  PV(l-e“DT)  >  f(V) 


0  otherwise 


and,  of  course,  t^  =  0. 

Now  we  assume  that  exactly  n  wells  must  be  drilled  in  the  interval 
[0,T].  Again,  the  concavity  of  f(v)  implies  that 


V1  “  ' 


=  v  =  V. 
n 


Equating  to  zero  the  derivatives  of  with  respect  to  t^,  i  =  l,...,n, 

we  can  show  that 

*  _  n  *  _  I  *  (n-l)T 

C1  '  U’  C2  "  n *  *  * " ’  n  n 

(The  above  is  true  subject  to  profitability). 

The  profit  function  is  evaluated  at  the  optimum  as  follows: 

J*  =  nPV(l  -  e~DT/n)  -  nf (V)  .  (15) 

The  circumstances  under  which  n  wells  are  more  profitable  than  (n-1)  wells 

*  * 

may  be  determined  by  comparing  with  the  condition 


•k  * 

j  -  j  .  >  o 

n  n-1 


(16) 
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reduces  to 


+  (n-l)e-DT/<”-1>  -  «e-ra/"  >  0 


(17) 


in  this  case. 

Define 


*  * 

Y  =  J  -  J  , 
n  n  n-1 


(18) 


The  following  properties  enable  us  to  propose  a  method  for  determining 


the  optimal  number  of  wells  n  .  Note  that 


lim  J  =  -  =o  • 
n 

n-*=° 


thus  we  may  be  certain  that  n  <  »  . 


Proposition  2 :  is  strictly  concave  in  n,  for  0  —  r.  <  °°  . 


Proof:  It  is  easily  shown  that 


2  * 
dZJ 

n 


dn 


PVD2T2  -DT/n 
- ^ — e  <  0- 

n 


Q.E.D. 


Proposition  3:  Y^  is  monotonically  decreasing  in  n,  for  1  s  n  < 


Proof:  The  result  is  a  direct  consequence  of  the  strict  concavity  of  J  . 

n 


Q.  E.  D. 


Note  that 


<  ®  . 


(19) 


Proposition  3,  in  conjunction  with  (19),  permits  us  to  conclude  that 


there  exists  some  value  of  n,  say  n  ,  satisfying 
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Y  *  iO,. 
n 

(20) 

Y  <  0. 

nx+l 

*  * 

Then  J  attains  a  maximum  at  n  =  n  ,  so  that  (20)  represents  the  condition 
n 

k 

for  optimality  of  n  . 

Condition  (16)  may  also  be  viewed  in  a  different  light.  Replacing  the 

inequality  with  an  equality,  we  may  solve  for  T  to  obtain  various  "points  of 

indifference"  T  *  for  different  values  of  n.  Suppose  T,  is  the  solution  for 
n  1 

* 

n  =  1;  then  may  be  interpreted  as  the  minimum  horizon  length  for  which  n  =  1 

k 

(or  equivalently,  the  maximum  horizon  length  for  which  n  =0).  These  points 
of  indifference  possess  the  following  property: 

k  k 

Proposition  4:  T  *  is  monotonically  increasing  in  n  ,  for  1  s  n  . 
n  dT  * 

Proof :  We  seek  to  show  that  — £  >  0.  Replacing  the  inequality  in  (17)  with 

dn  * 

an  equality  and  letting  T  =  T^*,  n  =  n  ,  we  implicitly  differentiate  Tn* 

k 

with  respect  to  n  to  obtain 


dT  * 

_ n  _ 

* 

dn 


1+DT  *\  -DT  */n*  f  1+  DV  )  °V(n  _1) 
n*S-  je  a  -  V  Tn^l)  1 


/  -DT  Vn 
(  n 
Vs 


-  e 


-DT  VCn^-l) 
n 


T 


Clearly  the  denominator  >0;  so  we  need  to  show  that  the  numerator  >  0.  Letting 

Z  .(l  +  21)  e-DT/n 

n  v.  n  /  ’ 


we  see  that 


dZn  D2!2  -DT/n  - 

-K"T-e  >  0  ' 

n 

completing  the  proof. 

Q  •  E  •  D  • 
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As  a  consequence  of  Proposition  4,  the  optimal  number  of  wells  n 
must  satisfy 


ls  sI<T4  .  (21) 

n  n  +1 


Case  2 :  f"(v)  >  0. 

* 

As  in  Section  3,  the  strict  convexity  of  f(v)  implies  that  v^  <  ®  , 

so  that  no  upper  bound  is  required  on  v^. 

Differentiating  J  with  respect  to  t  ,  v  ,  i  =  l,...,n,  we  can  show 
n  li 

that 


*  *  1  *  (n-l)T 

t,  =0,  t_  =  t  =  - 

1  2  n  n  n 


and  v. 


=  v  =  V  ,  where  V  is  the  solution  to 
n  n  n 


f' (V  )  =  P(1  -  e~DT/n) 
n 


(22) 


It  is  easily  shown,  as  in  Proposition  1,  that  Vq  >  0. 

*  * 

Substituting  for  t^  and  in  (14) ,  we  get 

J*  =  nPV  (1  -  e-DT/n)  -  nf (V  )  .  (23) 

n  n  n 

* 

Defining  Yn  as  in  (18),  we  may  use  (20)  or  (21)  to  determine  n  . 
Example: 

Suppose  T  =  20,  D  *  1,  P  B  100, 

Case  1:  f(v)  =  lOv,  V  ■  40 

We  first  evaluate 

Y  =  4000(0.9  +  (n-l)e-20^n~1^-ne  2°^n] , 
n 

* 

and  use  condition  (2)  to  obtain  n  ■  28. 

* 

Then  the  drilling  interval  *  0.526,  and  v^^  ■  40. 
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Case  2:  f(v)  *  10v2. 

As  before,  we  compute 

Yn*250  [n(l  -  e'20/n)2  -  (n-1)  (1  -  e"20/^"1))2] 

•ff 

and  use  condition  (20)  to  obtain  n  =16. 

ft 

Then  the  drilling  interval  =  1.25,  and  =  3.567. 


4.2  Infinite  Horizon  Models : 

It  is  clear  by  symmetry  that  the  interval  T  between  drillings  will 
remain  constant;  so  will  the  capacity  v  of  the  oil  well.  If  K  represents 
the  average  profit,  the  objective  is  to 


Maximize 

v,T 


/  PDve  Ptdt  -  f(v) 
0 


(24) 


The  above  expression  may  be  rewritten  as 


Pv(l  -  e_DT)  -  f(v) 


We  analyze  the  two  cases  as  follows : 
Case  1:  f'(v)  sO. 


(25) 


As  before,  the  concavity  of  f(v)  forces  us  to  impose  an  upper  bound 
on  v,  so  we  let  v  ^  V  <  « 


Then  the  optimal  oil  well  capacity  is  given  by 
* 


V 


if  PV(1  -  e"DT  )  >  f(V) 


(26) 


otherwise 


The  optimal  drilling  interval  is  determined  by  differentiating 
* 

to  T;  thus  T  is  the  solution  to 
* 


-DT  * 

a  (1  +  DT  ) 


f(V) 

PV  • 


K  with  respect 


(27) 
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Case  2:  f ' (v)  >  0. 

As  we  have  explained,  no  upper  bound  is  required  on  v,  as  a  consequence 
of  Che  convexity  of  f(v). 

The  optimal  values  are  obtained  by  differentiating  K  with  respect  to 

k  k 

v  and  T  respectively;  so  we  solve  for  v  and  T  from  the  equations 
f'(v*)  =  P(1  -  e~DT  )  (28) 


and 


■k  -k  k 

v  f 1  (v  )  —  f  (v  ) 
Dv*(P  -  f’(v*)) 


(29) 


It  is  easily  shown,  as  in  Proposition  1,  that  v  >0,  so  that  it  is 
always  optimal  to  drill.  Using  (28),  (29),  in  (25),  the  maximum  average  profit 
is  given  by 


v< 

K 


*  -DT 
PDv  e 


(30) 


jjf 

Note  that  K  is  equal  to  the  average  profit  obtained  by  selling  oil  at  the 

k 

* 

constant  rate  of  Dv  e  ,  with  no  drilling  costs. 


5 .  Stochastic  Models 

It  is  often  the  case  that  the  outcome  of  a  drilling  operation  is,  a  priori, 
unknown;  so  we  now  treat  the  capacity  v  of  the  oil  well  as  a  random  variable, 
instead  of  as  a  decision  variable.  Let  ?(v)  be  the  probability  density 
function,  $(•)  the  cumulative  probability  distribution,  and  v  the  expected 
value  of  the  random  variable  v.  We  assume  that  a  fixed  cost  Q  is  incurred  per 
drilling.  J  now  represents  the  expected  profit  function. 
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5.1  Finite  Horizon  Models : 

Single-well  Model: 

Here  we  admit  the  possibility  of  drilling  no  more  than  a  single  well  in 
the  interval  [0,T],  where  T  is  finite.  The  objective  is  to 

Maximize  =  E 

or  equivalently, 

=  vP(l  -  e  DT)  -  Q  ,  (31) 

where  v  is  the  expected  value  of  v.  It  is  obvious,  from  (31),  that  the 

* 

optimal  policy  is  to  drill  (at  t^  =  0)  iff 

vP(l  -  e_DT)  >  Q.  (32) 


/  PDve~Dtdt  -  Q 
0 


Two-well  Model: 

We  assume  that  no  more  than  two  wells  may  be  drilled  in  the  interval  [0,T], 
We  first  remark  that  consideration  of  the  two-well  case  implies  that  it  must  be 
profitable  to  drill  the  first  well;  that  is,  (32)  holds.  The  issue,  therefore, 
reduces  to  deciding  if,  and  when,  a  second  well  is  to  be  drilled,  contingent 
on  the  realization  of  the  initial  drilling,  which  we  shall  denote  by  w.  We 
then  seek  to 


Maximize  3^ 
t2 


I  PDwe"Dtdt  +  E 
0 


PDve-D(t~t2)dt 


2Q 


or  equivalently, 


-Dt,  _  -D(T-t 

Pw(l  -  e  J  +  Pv(l  -  e 


2) 


)  ~  2Q, 


(33) 


where  Z2  represents  the  time  at  which  the  second  well  is  drilled,  given  that  two 
wells  must  be  drilled  in  the  interval  [0,T].  Differentiating  (33)  with  respect 
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to  t£,  we  obtain  the  optimal  solution 


* 


1 

2 


_1 

2D 


(jun  v  -  . 


(34) 


Further  analysis  is  necessary  to  identify  the  best  course  of  action 

* 

depending  on  the  outcome  w  of  the  first  drilling.  Notice  that  t^  is  a 

monotonically  increasing  function  of  w.  Let  w^  and  represent  the  values 

*  * 

of  w  for  which  »  0  and  t£  =  T,  respectively  (see  Fig.  3).  Then  from 
(34), 


-  -DT 


w^  =  v  e 


and 


-  DT  . 
w^  =  v  e 


The  plot  of  t^  as  a  function  of  w  is  shown  below: 


(35) 


(36) 


iij 


We  now  analyze  several  cases : 


Case  1:  w  >  w^. 

Here  (3A)  and  (36)  together  imply  that  t*  >  T  (see  Fig.  3);  consequently, 
an  optimal  policy  will  not  include  a  second  well. 

Case  2:  w^  <  w  <  w^. 

It  is  easily  inferred  from  (34),  (35),  (36)  that  0  <  t*  <  T  (see 

& 

Fig.  3) ,  so  that  a  second  well  should  be  drilled  at  t2  if 
*  * 

J2  >  Ji  •  (35) 

■k  i k 

J^,  are  computed  based  on  the  realization  of  the  initial  drilling; 
thus  (31)  reduces  to 

J*  *  wP(l  -  e“DT)  -  Q,  (36) 

while  (33)  and  (34)  together  yield 

J*  =*  (w  +  v)P  -  2Vwv  e“DT/2  -  2 Q.  (37) 

Then  condition  (35)  may  be  restated  as  follows:  A  second  well  should  be  drilled 
* 

at  t2  if 


where 


w  <  w3  , 

w3  -  (vr  -  vw)z  eDT. 


(37) 


(38) 


Case  3:  w  <  w^. 

* 

Equations  (34)  and  (35)  together  imply  that  <  0  (see  Fig,  3);  then 

the  optimal  policy  is  to  abandon  the  first  well  and  redrill  immediately  (at 
* 

t2  *  0),  provided  this  action  enhances  the  expected  profit. 
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Treating  the  cost  incurred  in  the  initial  drilling  as  a  sunk  cost,  the  profit 

generated  by  retaining  the  first  well  throughout  the  interval  [0,T]  is  wP(l-e  ). 

* 

On  the  other  hand,  abandoning  the  first  well  and  drilling  anew  at  t^  =  0  yields 

-  -DT  - 

an  expected  profit  of  vP(l  -  e  )  -  Q.  The  latter  course  of  action  is  de¬ 
sirable,  therefore,  if 

-DT  -DT 

v  P(1  -  e  )  -  Q  >  wP(l  -  e  Ui) 

or  equivalently,  if 


w  <  w4. 


where 


w,  =  v  - 
4 


P(1  ~  e-°T) 


(39) 


(40) 


Note  that  w,,  is  larger  than  w^ ,  w^ ,  and  w^ ;  the  relative  magnitudes 
of  the  latter  three  quantities,  however,  depend  on  Q,  P,  and  T. 

The  analysis  is  summarized  below: 

1.  w  >  do  not  drill. 

* 

2.  w^  <  w  <  w^:  drill  at  t2  iff  w  <  w^, 

* 

where  t2  is  given  by  (34). 

* 

3.  w  <  w^:  drill  at  t2  *  0  iff  w  <  w^. 

The  values  of  w^,  w2,  w^,  and  w^  are  obtained  from  (35), 

(36),  (38),  and  (40)  respectively. 


Example: 


Let  T  -  5,  P  ■  100,  Q  -  1000,  v  -  15,  D  ■  1. 
First  note  that 


v  P(1  -  e~DT)  -  1489.89  >  Q  -  1000, 
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so  that  the  condition  for  drilling  the  first  well  is  satisfied. 

Using  (35),  (36),  (38),  and  (40),  we  get 

w^  ■  0.101 

w2  =  2226.2 

w^  =  74.96 

w.  =  4.93  . 

4 

Notice  that  w,  <  w.  <  w.  <  w„  . 

14  3  2 

Since  >  w^,  decision  rules  1  and  2  reduce  to: 

1.  w  >  74.96:  do  not  redrill. 

•k 

2.  0.101  <  w  <  74.96:  redrill  at  t2< 

Since  w.  >  w, ,  decision  rule  3  reduces  to: 

4  1 

3.  w  <  0.101:  redrill  immediately. 

Suppose  w  »  10 ;  then  the  optimal  policy  is  to  drill  a  second  well  at 

t*  =  2.297  (from  (34)). 

5 . 2  Infinite  Horizon  Model: 

Assume  a  continuous  discount  rate  r  >  0. 

The  depletion  of  oil  reserves  in  the  well  is  accompanied  by  a  decrease 
in  the  rate  of  oil  extraction,  so  that  a  stage  is  reached  when  it  becomes  de¬ 
sirable  to  abandon  the  current  well  and  drill  a  new  one.  The  state  of  the  process 
is  completely  determined  oy  the  oil  reserve  C.  Thus  there  exists  a  level  C 
such  that  the  optimal  policy  assumes  the  following  form: 

Retain  the  current  well  if  C  >  C, 

Drill  a  new  well  if  C  s  C. 


In  view  of  the  random  nature  of  v,  the  above  policy  has  the  following  implica¬ 
tions:  if  v  ^  Cj  the  well  is  abandoned  and  a  new  one  drilled  immediately; 

_Dt  A 

while  if  v  >  C,  the  well  is  operated  until  ve  =  C,  where  t  is  the  length 
of  time  for  which  the  well  has  been  in  operation.  We  seek  an  expression  for 


Figure  4 


Let  J(C)  be  the  value  function  representing  the  expected  profit  from  t 
Co  infinity,  given  that  the  oil  reserve  at  time  t  is  C.  It  is  easy  to  see 
that 


J(C)  »  J(C)  for  CsC,  (41) 

since  the  optimal  policy  involves  redrilling  whenever  C  s  C.  On  the  other 

hand,  suppose  C>C  (see  Fig.  4).  Then  the  quantity  of  oil  in  the  well  will 

decrease  from  C  to  Ce”D5t  in  the  interval  [t,  t  +  <5t]  (from  (3)),  generating 

“D6t 

a  profit  of  PC(1  -  e  ).  Thus  we  may  write 


o> 


-20- 


J (C)  »  PC(1  -  e"D6t)  +  e“r6t  J(Ce"D,St)  .  (42) 

Using  the  Taylor  series  expansion  for  e,  writing 

J(Ce_D<St)  «J(C  -  CDot)  ^JCC)  -  J'(C)CD6t, 

and  omitting  the  square  and  higher  powers  of  5t,  (42)  may  be  reduced  to  the 
following  differential  equation: 

J'(C)  +  2*^^  *  P,  (43) 

where  p  =  r/D. 

Using  CP  as  the  integrating  factor,  (43)  may  be  solved  to  obtain 

J(C)  =  tp+it  c  +  KC_P  ’  (44) 

where  K  is  a  constant  of  integration. 

When  C  decreases  to  C,  the  well  is  abandoned  and  a  new  one  drilled. 
Conditioning  on  the  outcome  of  the  drilling,  we  may  write 

J(C)  -  Prob(v>C)  [J(v)-Q]  +  .'rob (v<C)  [ J(C)  -  Q]  ,  (45) 

which  simplifies  to 

OO 

J(C)  =>  - — —  [  /  . 9(v) J(v)dv  -  Qj  .  (46) 

1  -  i(c)  c 

Note  that  (46)  holds  only  for  C  >  0;  if  C  =  0,  then  the  optimal  policy 
involves  only  a  single  drilling  (at  t  ■  0) ,  so  that 

00  — 

J(C)  -  /  PDve"Dte"rtdt  -  Q  -  (^)  -  Q,  for  C  •  0. 


If  the  above  expression  is  negative,  that  is,  if 


Pv 

(1+P) 


<  Q  , 


(47) 


then  it  will  never  be  optimal  to  drill. 

Our  objective  here  is  to  maximize  the  value  function  at  t  *  0.  Assuming 
that  there  is  no  functioning  oil  well  at  the  beginning  of  the  process,  it  follows 
from  (41)  that  the  objective  is  to 


Maximize  J(C)  , 
C 


where  J(C)  is  given  by  (46).  To  evaluate  (46),  we  substitute  for  J(v) 
from  (44);  but  first  we  need  an  expression  for  the  constant  of  integration  K. 

As  a  boundary  condition,  we  require  that  J(C),  evaluated  at  C  from  (44), 
equals  J(C)  from  (46),  so  that  J(C)  is  continuous  at  C.  We  now  demonstrate 
intuitively  that  the  boundary  condition  we  have  just  imposed  is  entirely 
appropriate. 

Suppose  J(C)  takes  a  downward  jump  at  C  (see  Fig.  5).  Then  we  are 
clearly  better  off  by  increasing  C,  and  we  can  keep  doing  this  until  the  two 
parts  of  J(C)  intersect. 


Figure  5 
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On  the  other  hand,  suppose  J(C)  takes  an  upward  jump  at  C  (see  Fig.  6). 

Then  we  can  increase  profits  by  decreasing  C,  and  this  process  continues  until 
the  two  parts  of  J(C)  intersect. 


As  a  consequence  of  the  assumed  boundary  condition,  we  have 

r  r  \ 


J<£>  -  feliy  c  +  kc-» 


1-<KC) 


l„9(v)  (y"y  v+Kv~°)  dv  -Q 


from  which  we  solve  for  K  as  a  function  of  C: 


K(C)  = 


P  / „  9(v)(v-C)dv  -  Q 

(p+1)  C _ 

00 

9(v) (C  p-  v  °)  dv 
C 


(48) 


It  is  easily  verified  from  (48)  that  K(0)  =  0,  and  Lim  K(C) 

C-*» 


»  —  00. 


Now  K(C)  merely  represents  the  value  of  the  constant  of  integration  K 
which  equates  J(C),  evaluated  at  C  from  (44),  with  J(C)  from  (46).  Further, 
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J (C)  is  increasing  in  K  (from  (44)).  Thus,  if  J(C)  attains  a  maximimum 
at  C  ,  it  is  clear  that  K(C)  must  increase  with  respect  to  C  in  [0,C  ], 
and  decrease  thereafter;  in  other  words,  K(C)  also  attains  a  maximum  at  C 
(see  Fig.  7). 

Since  the  objective  is  to  maximize  J(C),  or  equivalently,  K(C) ,  with 
respect  to  C,  we  differentiate  (48)  with  respect  to  C  and  equate  it  to  zero 
accordingly,  C  is  obtained  as  the  solution  to 


/  r(v)(v-CX)dv  -  — I  9 (v)(C*“p-  v~P)dv  =  Q(g+1)  (49) 


p  "  C* 


Differentiating  (48)  twice  with  respect  to  C,  and  using  (49),  we  may 
show  that 


K"(C*) 


PD 


<  0  , 


C*  /„f(v)(C*~P-v~p)dv 


so  that  the  condition  for  a  maximum  is  satisfied. 


K(C  )  is  evaluated  using  (48)  and  (49) 


-* 


«c*)  - 

P (1+p) 


(50) 


The  value  of  the  objective  function  at  the  optimum  is  obtained  from 
(44)  and  (50) : 


PC 

J(c  )  =  Zf- 

p 


(51) 


Equation  (51)  affords  an  interesting  interpretation.  Note  that 

f  PDc\'rtdt  =  ^  . 

0  0 


Thus  the  optimal  choice  of  C  for  the  original  problem  yields  an  infinite 
horizon  discounted  profit  which  is  exactly  equal  to  that  derived  from  a  constant 
supply  of  oil  at  the  rate  DC  ,  with  no  drilling  costs;  further,  this  constant 
rate  is  precisely  the  rate  of  oil  extraction  from  the  well  at  the  instant  prior 
to  abandonment. 

We  show  diagrammatically  the  relationship  between  J(C),  J(C),  and  K(C) 


Figure  7 


Finally,  we  study  the  effect  of  variation  of  different  parameters  on  C  . 


Representing  the  left  hand  side  of  (49)  by  F,  we  can  show  that 


By  the  implicit  function  theorem,  we  have 

3C*  _  3F/3p 

3p  3F/3C* 


which,  in  conjunction  with  (54)  and  (55)  ,  yields 


<  i 

I 


3C 

3o 


>  0. 


(58) 


From  (56),  (57),  and  (58),  we  conclude  chat  the  optimal  redrilling  level 


n  > 
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For  P  *  1000,  Q  =  100,  r  ■  0.1,  D  =  1,  the  above  equation  reduces  to 


«*0.i  « * 

C-LC—  -.0^9.  --Q-9]  +I(1-  cV  =  o  n 

0.09  2  u  L  '  ••L■L, 


which  may  be  solved  to  yield  C  =  0.282. 


6 .  Concluding  Remarks 

The  models  developed  in  this  paper  appear  to  permit  a  fairly  wide  range 
of  practical  applications.  Situations  calling  for  slightly  different  sets  of 
assumptions  could  easily  be  analyzed  in  similar  fashion. 
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A  number  of  real  workd  processes  can  be  modelled  as  exponential  decay  1 
processes.  Examples  are:  machine  replacement,  oil  well  extraction,  adver-  j 
tising  goodwill,  repair  and  cleaning  activities,  etc..  In  this  paper  we  ! 
analyze  a  series  of  discounted  or  undiscounted,  determinisitic  or  sto^Mstic  j 
exponential  decay  models.  We  characterize  finite  and  infinite  horizon  optimal- 
solutions  for  each  model.  We  show  that  the  solution  can  be  characterized  for  j 
the  oil  drillers  problem  in  the  following  way:  Once  a  well  of  sufficient  j 
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capacity  is  drilled ,  oil  is  pumped  from  it  until  the  oil  remaining  decreases 
to  a  fixed  cut-off  level;  then  the  well  is  abandoned,  and  a  new  veil  is  drilled 
The  resulting  process  when  raoeated  over  time  apoaars  to  be  the  same  as  an  oil 
source  which  produces  oil  revenue  continuously  at  the  fixed  cut-off  level.  In 
ocher  words,  the  excess  revenue  received  from  an  oil  veil  when  ics  caoacicv  is 
greater  than  the  cutoff  level  is  jusc  sufficient  to  pay  for  drilling  costs  for 
a  new  well. 


